The environment understanding interface: detecting and tracking human activity through multimedia sensors

A technique for using economical multimedia sensors to autonomously track human beings is presented. A sequence of color images captured from a video camera is processed in real-time to determine target locations. This data may be used to guide a computer-controlled pantilt-zoom camera, and may be fused with sound information to determine the location of a person speaking. Such capabilities are the foundation of what we call the Environment Understanding Interface, a new paradigm for human-computer interaction. Applications include automated videoconferencing, security/surveillance, home automation, and assistance for the disabled. A prototype of this system has been implemented on an ordinary multimedia PC running OS/2.